Three-dimensional interface control method and terminal

ABSTRACT

Embodiments of this disclosure disclose an interface control method, and relate to the field of virtual reality technologies, and are applied to a virtual reality device, an augmented reality device, a mixed reality device, or the like. The method in embodiments of this disclosure includes: dividing, based on a user operation, an interface that includes dividable components and that is displayed in a virtual environment, and separately displaying and controlling subinterfaces obtained through division. Based on the interface control method, a user may divide the interface including the dividable components based on a user requirement, so that interface display flexibility and user experience are improved.

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure is a continuation of International Patent ApplicationNo. PCT/CN2021/095952, filed on May 26, 2021, which claims priority toChinese Patent Application No. 202010565198.3, filed on Jun. 19, 2020.The disclosures of the aforementioned applications are herebyincorporated by reference in their entireties.

TECHNICAL FIELD

This disclosure relates to the field of virtual reality technologies,and in particular, to a three-dimensional interface control method and aterminal.

BACKGROUND

In a scenario such as virtual reality (VR), augmented reality (AR), ormixed reality (MR), a terminal provides interactive immersive experiencefor a user by constructing a virtual environment. In the virtualenvironment, the terminal interacts or exchanges information with theuser through a three-dimensional user interface (3D UI). The 3D UI is amedium for direct interaction or information exchange between a systemand the user in a three-dimensional environment. With the development ofvirtual reality technologies, the 3D UI is widely used in VR, AR, and MRproducts, covering various fields such as office, audio and video,entertainment, and industry.

A separate interface refers to separate subinterfaces formed by dividingdifferent functional areas of a same application in space, and there isa gap between different subinterfaces. The separate interface is acommon design and layout manner in the 3D UI, and can fully use infinitedisplay space in the virtual environment, and help the applicationdistinguish functional areas, such as a display area or an operationarea, based on an operation habit of the user.

However, an existing dividable interface has a preset layout, and eachsubinterface is displayed in a fixed location. When the user moves, asubinterface corresponding to the operation area may be far away fromthe user, and user experience is poor.

SUMMARY

An embodiment of this disclosure provides an interface control method todivide an interface into subinterfaces for display based on a useroperation, improving interface display flexibility and user experience.

A first aspect of an embodiment of this disclosure provides an interfacecontrol method. The method includes: A terminal obtains a firstoperation performed by a user on a first interface displayed in avirtual environment; if the first interface includes at least twodividable components, the terminal obtains preset division locationinformation on the first interface, where the division locationinformation indicates a boundary between two adjacent dividablecomponents on the first interface; and if the first operation meets apreset condition, the terminal displays, in the virtual environmentbased on the first operation and location information of a divisionline, a first subinterface and a second subinterface that are obtainedby dividing the first interface, where the first subinterface includes afirst dividable component of the at least two dividable components, andthe second subinterface includes a second dividable component adjacentto the first dividable component.

The interface control method is applied to a VR, AR, or MR product. Forthe interface displayed in the virtual environment, if the interfaceincludes at least two dividable components, division locationinformation between the dividable components may be preset. In responseto the first operation obtained by the terminal on the first interfacedisplayed in the virtual environment, the terminal may divide theinterface into at least two subinterfaces respectively for display andcontrol based on the preset division location information. Based on theinterface control method, the user may divide the interface includingthe dividable components based on a user requirement, so that interfacedisplay flexibility and user experience are improved.

In an embodiment of the first aspect, the first operation includes firstvoice information obtained by the terminal by using a voice collectionapparatus, or a first gesture obtained by the terminal by using a sensorsystem.

In the method, the user operation may be voice information or a gesture.In other words, the user divides the interface by using a voice or agesture. The operation is convenient.

In an embodiment of the first aspect, the first gesture includes: movinga flat palm with fingers together by a preset distance in a firstdirection in which a palm plane extends, where the first direction inwhich the palm plane extends is parallel to the boundary.

The method provides a specific division gesture. The user moves a handalong the palm plane, to provide information of an expected divisionmanner to the terminal. The gesture is simple and complies with a userhabit, and operation efficiency can be improved.

In an embodiment of the first aspect, the method further includes: Theterminal displays the division line on a boundary between the firstdividable component and the second dividable component.

In the method, the terminal displays the division line in a divisionlocation of the interface, so that a division guide can be provided forthe user, and operation efficiency can be improved.

In an embodiment of the first aspect, the displaying, by the terminal inthe virtual environment based on division location information inresponse to the first operation, a first subinterface and a secondsubinterface that are obtained by dividing the first interfacespecifically includes:

The terminal displays the first subinterface in a first location, anddisplays the second subinterface in a second location, where the firstlocation is the same as a first initial location, a distance between thesecond location and a second initial location in a direction away fromthe first initial location is a first preset value, the first initiallocation is a location of the first subinterface on the first interfacewhen the terminal obtains a first instruction, and the second initiallocation is a location of the second subinterface on the first interfacewhen the terminal obtains the first instruction; or the second locationis the same as a second initial location, and a distance between thefirst location and a first initial location in a direction away from thesecond initial location is a second preset value; or a distance betweenthe first location and a first initial location in a direction away froma second initial location is a third preset value, and a distancebetween the second location and the second initial location in adirection away from the first initial location is a fourth preset value.

This method provides various forms of locations displayed after theinterface is divided into the subinterfaces, where some subinterfacesmay be moved, or the subinterfaces may be away from each other. Thismethod provides various separation forms, and improves flexibility ofimplementing the solution. In addition, a relative distance between thesubinterfaces obtained through division is increased, and information ofdivision completion may be fed back to the user, so that the usersubsequently separately controls the subinterfaces, and user experiencecan be improved.

In an embodiment of the first aspect, the method further includes: Theterminal moves the first subinterface in response to a second operationon the first subinterface. Optionally, the second operation may includeone or more gestures. Optionally, the more gestures include a gesturefor the terminal to select the first subinterface and a gesture for thefirst subinterface to follow.

In this method, a method for moving a subinterface based on the useroperation is provided. Because the subinterface may be independentlymoved based on the user operation, in a scenario in which the usermoves, the subinterface may continue to keep a close distance from theuser, so that user experience can be improved.

In an embodiment of the first aspect, the second operation includes: apinch grip gesture pointing to the first subinterface, where the pinchgrip gesture includes a hand state in which a thumb approaches at leastone of four fingers, and keeping the pinch grip gesture and moving; ortouching and holding the first subinterface and moving; or keeping anopen palm gesture and moving, where the open palm gesture includes ahand state with fingers together and a palm up.

In this method, various specific manners of moving the subinterface bythe user by using a gesture are provided, including moving by using apinch grip gesture, moving by using a touch gesture, or moving by usingan open palm gesture. Because display space in the virtual environmentis huge, when the user moves the interface in a short distance, agesture such as the pinch grip gesture or the touch gesture may be used.In a scenario in which the interface is moved in a long distance, it isdifficult to maintain a pinch grip or touch state of the interface. Inthis scenario, the interface may follow the open palm gesture. Because adistance between the user gesture and the interface is not limited whenthe interface follows, an operation is more convenient. This methodprovides various manners of moving an interface, and flexibility ishigh.

In an embodiment of the first aspect, the method further includes: Theterminal moves the first subinterface; and if a distance between thefirst subinterface and the second subinterface is less than a firstthreshold, the terminal displays, in the virtual environment, a secondinterface obtained by combining the first subinterface and the secondsubinterface, where a layout of the second interface is the same as ordifferent from a layout of the first interface.

This method provides a method of combining and restoring the dividedsubinterfaces, and improves completeness of implementing the solution.In this method, the terminal may restore an original interface layoutduring combination and restoration based on before-division locationinformation of the subinterface on the first interface. In addition, alayout of the combined interface may be different from a before-divisionlayout of the first interface. In other words, according to a manner inwhich the subinterfaces are close to each other when the user combinesthe subinterfaces, the layout of the second interface is determined, sothat diversity of interface layouts is improved.

In an embodiment of the first aspect, the method further includes: Theterminal displays a third interface in the virtual environment, wherethe first interface is an interface of a first application, and thethird interface is an interface of a second application; the terminalmoves the first subinterface and/or the third interface; and if adistance between the first subinterface and the third interface is lessthan a second threshold, the terminal displays, in the virtualenvironment, a fourth interface obtained by combining the firstsubinterface and the third interface.

The method provides a cross-application interface combination displaymanner. In some application scenarios, the user may combine thesubinterface of the first application and the interface of the secondapplication for display. For example, the user may combine a lyricsdisplay interface of a music application and an MV playback picture of avideo application for display, so that the user can simultaneouslyexperience functions of a plurality of applications.

In an embodiment of the first aspect, the method further includes: Theterminal displays a third subinterface and a fourth subinterface in thevirtual environment, where the third subinterface and the fourthsubinterface are obtained by dividing the third interface, the firstinterface is the interface of the first application, and the thirdinterface is the interface of the second application; the terminal movesthe first subinterface and/or the third subinterface; and if a distancebetween the first subinterface and the third subinterface is less than athird threshold, the terminal displays, in the virtual environment, afifth subinterface obtained by combining the first subinterface and thethird subinterface.

The method provides a cross-interface combination display manner. Insome application scenarios, the user may combine the subinterface of thefirst application and the subinterface of the second application fordisplay. Specifically, the two subinterfaces are moved, and when thedistance between the two subinterfaces is less than the threshold, thetwo subinterfaces are combined into one subinterface. Optionally, thecontrol interface of the first application and the control interface ofthe second application are combined into one subinterface. Only one iconcorresponding to a same control function is reserved on the controlinterface of the first application and the control interface of thesecond application. An icon of a control function unique to the controlinterface of the first application relative to the control interface ofthe second application is reserved in the combined subinterface.

In an embodiment of the first aspect, the first subinterface includes afirst icon for controlling a first function of the first application.The third subinterface includes a second icon for controlling a secondfunction of the second application. The first function is the same asthe second function. The fifth subinterface includes a third icon. Thethird icon is used to control the first function of the firstapplication and the second function of the second application.

In this method, a method for combining subinterfaces of differentapplications and combining and displaying icons corresponding to a samefunction is specifically described. This can reduce occupancy on adisplay interface of the virtual environment and avoid repeated displayof a same function icon of a plurality of applications.

In an embodiment of the first aspect, the first subinterface is an inputinterface of the first application. The second subinterface is an outputinterface of the first application. The third subinterface is an inputinterface of the second application. The fourth subinterface is anoutput interface of the second application. The method further includes:The terminal displays an indication identifier on an edge of the fifthsubinterface, where the indication identifier is used to determine anapplication currently controlled by the fifth subinterface; when theindication identifier points to the second subinterface, the terminalcontrols the first application based on an input operation performed onthe fifth subinterface; and when the indication identifier points to thefourth subinterface, the terminal controls the second application basedon an input operation performed on the fifth subinterface. The inputinterface is an interface on which the terminal obtains an input of theuser, and the output interface is an interface on which the terminaloutputs information to the user. Optionally, the input interfaceincludes a control interface, and the output interface includes a videoimage display interface or a lyrics display interface.

This solution provides a method for controlling a plurality ofapplications. The two applications may be separately controlled in anaimed manner on the fifth subinterface obtained by combining the inputsubinterfaces of the first application and the second application. Thisimproves utilization efficiency of an input control area, and avoidsthat the user needs to change an input interface to separately controlan application when simultaneously using the plurality of applications,operation is convenient, and user experience is good.

A second aspect of an embodiment of this disclosure provides a terminal.The terminal includes: an obtaining unit, configured to obtain a firstoperation on a first interface displayed in a virtual environment, wherethe first interface includes at least two dividable components; and adisplay unit, configured to display, in the virtual environment based ondivision location information in response to the first operation, afirst subinterface and a second subinterface that are obtained bydividing the first interface, where the first subinterface includes afirst dividable component of the at least two dividable components, thesecond subinterface includes a second dividable component adjacent tothe first dividable component, and the division location informationincludes a boundary between the first dividable component and the seconddividable component.

In an embodiment of the second aspect, the first operation includesfirst voice information obtained by the terminal by using a voicecollection apparatus, or a first gesture obtained by the terminal byusing a sensor system.

In an embodiment of the second aspect, the first gesture includes:moving a flat palm with fingers together by a preset distance in a firstdirection in which a palm plane extends, where the first direction inwhich the palm plane extends is parallel to the boundary.

In an embodiment of the second aspect, the display unit is furtherconfigured to display a division line on a boundary between the firstdividable component and the second dividable component.

In an embodiment of the second aspect, the display unit is specificallyconfigured to:

display the first subinterface in a first location, and display thesecond subinterface in a second location.

The first location is the same as a first initial location, a distancebetween the second location and a second initial location in a directionaway from the first initial location is a first preset value, the firstinitial location is a location of the first subinterface on the firstinterface when the terminal obtains a first instruction, and the secondinitial location is a location of the second subinterface on the firstinterface when the terminal obtains the first instruction; or

the second location is the same as a second initial location, and adistance between the first location and a first initial location in adirection away from the second initial location is a second presetvalue; or

a distance between the first location and a first initial location in adirection away from a second initial location is a third preset value,and a distance between the second location and the second initiallocation in a direction away from the first initial location is a fourthpreset value.

In an embodiment of the second aspect, the terminal further includes:

a processing unit, configured to move the first subinterface in responseto a second operation on the first subinterface.

In an embodiment of the second aspect, the second operation includes: apinch grip gesture pointing to the first subinterface, where the pinchgrip gesture includes a hand state in which a thumb approaches at leastone of four fingers, and keeping the pinch grip gesture and moving; or

touching and holding the first subinterface and moving; or

keeping an open palm gesture and moving, where the open palm gestureincludes a hand state with fingers together and a palm up.

In an embodiment of the second aspect, the terminal further includes:

a processing unit, configured to move the first subinterface.

The display unit is further configured to: if a distance between thefirst subinterface and the second subinterface is less than a firstthreshold, display, in the virtual environment, a second interfaceobtained by combining the first subinterface and the secondsubinterface, where a layout of the second interface is the same as ordifferent from a layout of the first interface.

In an embodiment of the second aspect, the terminal further includes:

The display unit is further configured to display a third interface inthe virtual environment, where the first interface is an interface of afirst application, and the third interface is an interface of a secondapplication.

The processing unit is further configured to move the first subinterfaceand/or the third interface.

The display unit is further configured to: if a distance between thefirst subinterface and the third interface is less than a secondthreshold, display, in the virtual environment, a fourth interfaceobtained by combining the first subinterface and the third interface.

In an embodiment of the second aspect, the display unit is furtherconfigured to display a third subinterface and a fourth subinterface inthe virtual environment, where the third subinterface and the fourthsubinterface are obtained by dividing the third interface, the firstinterface is the interface of the first application, and the thirdinterface is the interface of the second application.

The processing unit is further configured to move the first subinterfaceand/or the third subinterface.

The display unit is further configured to: if a distance between thefirst subinterface and the third subinterface is less than a thirdthreshold, display, in the virtual environment, a fifth subinterfaceobtained by combining the first subinterface and the third subinterface.

In an embodiment of the second aspect, the first subinterface includes afirst icon for controlling a first function of the first application.The third subinterface includes a second icon for controlling a secondfunction of the second application. The first function is the same asthe second function.

The fifth subinterface includes a third icon. The third icon is used tocontrol the first function of the first application and the secondfunction of the second application.

In an embodiment of the second aspect, the first subinterface is aninput interface of the first application. The second subinterface is anoutput interface of the first application. The third subinterface is aninput interface of the second application. The fourth subinterface is anoutput interface of the second application.

The display unit is further configured to display an indicationidentifier on an edge of the fifth subinterface, where the indicationidentifier is used to determine an application currently controlled bythe fifth subinterface.

The terminal further includes:

a control unit, configured to: when the indication identifier points tothe second subinterface, control the first application based on an inputoperation performed on the fifth subinterface.

The control unit is further configured to: when the indicationidentifier points to the fourth subinterface, control, by the terminal,the second application based on an input operation performed on thefifth subinterface.

A third aspect of an embodiment of this disclosure provides a terminal.The terminal includes one or more processors and a memory. The memorystores computer-readable instructions. The one or more processors readthe computer-readable instructions in the memory, and the terminal isenabled to implement the method according to any one of the first aspector the possible implementations.

In an embodiment of the third aspect, the terminal includes a virtualreality device, an augmented reality device, or a mixed reality device.

In an embodiment of the third aspect, the terminal includes ahead-mounted display device.

A fourth aspect of embodiments of this disclosure provides a computerprogram product including instructions. When the computer programproduct runs on a computer, the computer is enabled to perform themethod according to any one of the first aspect and the possibleimplementations.

A fifth aspect of embodiments of this disclosure provides acomputer-readable storage medium including instructions. When theinstructions are run on a computer, the computer is enabled to performthe method according to any one of the first aspect and the possibleimplementations.

According to a sixth aspect, an embodiment of this disclosure provides achip, including a processor. The processor is configured to read andexecute a computer program stored in a memory, to perform the methodaccording to any possible implementation of any one of the foregoingaspects. Optionally, the chip includes the memory, and the memory andthe processor are connected to the memory by using a circuit or a wire.Further, optionally, the chip further includes a communicationinterface, and the processor is connected to the communicationinterface. The communication interface is configured to receive data orinformation or both that need to be processed. The processor obtains thedata and/or the information from the communication interface, processesthe data and/or the information, and outputs a processing result throughthe communication interface. The communication interface may be aninput/output interface.

For technical effects brought by any embodiment of the second aspect tothe sixth aspect, refer to technical effects brought by correspondingembodiments of the first aspect. Details are not described herein again.

It may be learned from the foregoing technical solutions that thisembodiment of this disclosure has the following advantages: According tothe interface control method provided in this embodiment of thisdisclosure, for the interface displayed in the virtual environment, ifthe interface includes at least two dividable components, the terminalpresets division location information between the dividable components,and in response to the first operation of the user, the terminal maydivide the interface into at least two subinterfaces respectively fordisplay and control based on the preset division location information.Based on the interface control method, the user may divide the interfaceincluding the dividable components based on requirements in differentapplication scenarios, so that interface display flexibility and userexperience are improved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram of a system architecture according to an embodimentof this disclosure;

FIG. 2 is a schematic diagram of a dividable three-dimensional interfaceaccording to an embodiment of this disclosure;

FIG. 3 is a schematic diagram of a system architecture of athree-dimensional interface control method according to an embodiment ofthis disclosure;

FIG. 4 a is a schematic diagram of an embodiment of a three-dimensionalinterface division method according to an embodiment of this disclosure;

FIG. 4 b is a schematic diagram of another embodiment of athree-dimensional interface division method according to an embodimentof this disclosure;

FIG. 5 a is one schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 b is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 c is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 d is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 e is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 f is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 g is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 5 h is another schematic diagram of three-dimensional interfacedivision according to an embodiment of this disclosure;

FIG. 6 is a schematic diagram of an embodiment of a three-dimensionalinterface movement method according to an embodiment of this disclosure;

FIG. 7 a is one schematic diagram of three-dimensional interfacemovement according to an embodiment of this disclosure;

FIG. 7 b is another schematic diagram of three-dimensional interfacemovement according to an embodiment of this disclosure;

FIG. 7 c is another schematic diagram of three-dimensional interfacemovement according to an embodiment of this disclosure;

FIG. 7 d is another schematic diagram of three-dimensional interfacemovement according to an embodiment of this disclosure;

FIG. 7 e is another schematic diagram of three-dimensional interfacemovement according to an embodiment of this disclosure;

FIG. 8 is a schematic diagram of an embodiment of a three-dimensionalinterface combination method according to an embodiment of thisdisclosure;

FIG. 9 a is one schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 9 b is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 10 is a schematic diagram of an embodiment of a combination methodof three-dimensional subinterfaces of a plurality of applicationsaccording to an embodiment of this disclosure;

FIG. 11 a is one schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 b is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 c is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 d is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 e is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 f is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 g is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 11 h is another schematic diagram of three-dimensional interfacecombination according to an embodiment of this disclosure;

FIG. 12 is a schematic diagram of an embodiment of a terminal accordingto an embodiment of this disclosure;

FIG. 13 is a schematic diagram of another embodiment of a terminalaccording to an embodiment of this disclosure;

FIG. 14 is a schematic diagram of an HMD device according to anembodiment of this disclosure; and

FIG. 15 is a block diagram of a software structure of a terminalaccording to an embodiment of this disclosure.

DESCRIPTION OF EMBODIMENTS

Embodiments of this disclosure provide an interface control method, toimplement flexible control of a three-dimensional interface.

The following describes embodiments of this disclosure with reference tothe accompanying drawings. It is clear that the described embodimentsare merely some but not all of embodiments of this disclosure. A personof ordinary skill in the art may learn that as a technology evolves anda new scenario emerges, technical solutions provided in embodiments ofthis disclosure are also applicable to a similar technical problem.

In the description, claims, and accompanying drawings of thisdisclosure, the terms “first”, “second”, and the like are intended todistinguish between similar objects but do not necessarily indicate aspecific order or sequence. It should be understood that the data termedin such a way are interchangeable in proper circumstances so thatembodiments of the present invention described herein can be implementedin other orders than the order illustrated or described herein.Moreover, the terms “include”, “contain” and any other variants mean tocover the non-exclusive inclusion, for example, a process, method,system, product, or device that includes a list of steps or modules isnot necessarily limited to those modules, but may include other modulesnot expressly listed or inherent to such a process, method, system,product, or device. Names or numbers of steps in this disclosure do notmean that the steps in the method procedure need to be performed in atime/logical sequence indicated by the names or numbers. An executionsequence of the steps in the procedure that have been named or numberedcan be changed based on a technical objective to be achieved, providedthat same or similar technical effects can be achieved.

First, some technical terms used in embodiments of this disclosure arebriefly described.

1. A virtual reality (VR) technology uses a computer to generate asimulated environment, so that a user is immersed in a virtualenvironment. The virtual reality technology uses data in real life andelectronic signals generated by computer technologies to combine theelectronic signals with various output devices, so that the electronicsignals are converted into phenomena that can be perceived by people.These phenomena may be true objects in real life or substances that areinvisible to naked eyes, and are represented by using athree-dimensional model. Because these phenomena are not directlyvisible to us, but the real world simulated by computer technologies,they are called virtual reality.

2. An augmented reality (AR) technology is a technology that cleverlyintegrates virtual information with the real world. A plurality oftechnical means such as multimedia, three-dimensional modeling,real-time tracking and registration, intelligent interaction, andsensing are widely used to simulate virtual information such as a text,an image, a three-dimensional model, music, and a video that aregenerated by a computer. Then, simulated information is applied to thereal world, and the two types of information complement each other, toimplement “augmentation” of the real world.

3. Mixed reality (MR) is a further development of VR and ARtechnologies, and introduces real-world scenario information intovirtual environments to build a bridge between the virtual world, realworld, and users for information interaction and feedback, so that userexperience is enhanced. Mixed reality generally uses an opticalperspective technology to superimpose virtual images on human eyes.

4. Mediated reality

The VR technology displays pure virtual digital images. The mixedreality (including AR) is virtual digital images +naked-eye reality,while the mediated reality is digital reality+virtual digital images.

In a scenario such as VR, AR, or MR, both a real world simulated byusing a computer technology and virtual information such as a text, animage, a three-dimensional model, music, and a video generated by acomputer are simulated, and all of the virtual scenarios involve adigital virtual scenario. The virtual scenario is usually implemented byusing a VR device, an AR device, or an MR device. To provide interactiveimmersive experience for a user, physical hardware such as a handle or atouchpad, or a system such as a head aiming or a gesture operation maybe used to perform an interface control operation, and the interfacecontrol operation is used as user input of the device. In a specificimplementation process of performing interface control by using agesture operation, a gesture operation (including a gesture and locationinformation of the gesture) of the user is obtained by using a sensorsystem of the device. In addition, generally a gesture of a user andlocation information of the gesture are variable in real time. A gestureobtained by a sensor system is mapped to a field of view of a device. Inother words. a gesture operation in a real world is mapped to a statusand a location movement of an operation icon (such as a hand or anarrow) in a virtual scenario, and the status and the location movementare displayed in real time on a display interface of the device, so thatthe user clearly perceives a real-time operation status of the user on athree-dimensional interface in the virtual scenario, and the mappingbetween the real world and the location in the virtual scenario is aconventional technology. Details are not described herein again.

This disclosure is mainly applied to a VR device, an AR device, or an MRdevice, for example, a head mounted display (HMD). A device usuallysupports a plurality of applications, such as a word processingapplication, a telephone application, an email application, an instantmessage application, a photo management application, a network browsingapplication, a digital music player application, and/or a digital videoplayer application. The applications are briefly referred to as anapplication in subsequent embodiments, and a specific type of theapplication is not limited. An interface of an application displayed inthe virtual environment is a three-dimensional interface. The interfaceis briefly referred to as an interface in embodiments of thisdisclosure.

In an existing three-dimensional interface division method, asubinterface is displayed based on a preset layout. Because asubinterface that is displayed separately occupies more space, generallyonly one separate interface application can be displayed in a visiblearea of a user, and a multi-application and multi-task use scenario ofthe user is limited. For example, when the user wants to use a dividablemusic interface and a dividable modeling application interface at thesame time, the two application interfaces cannot be displayed in onevisible area at the same time. If the two application interfaces aredisplayed in the same visible area, a problem of displaying functionareas of different applications may occur. As a result, the user cannotclearly identify the interface to which a specific function areabelongs. In addition, the divided operation area is displayed in a fixedlocation. When the user moves, a subinterface corresponding to theoperation area may be far away from the user, and user experience ispoor.

The interface control method provided in embodiments of this disclosuremay be applied to a terminal device such as a VR device, an AR device,or an MR device. The three-dimensional interface of the application canbe displayed through an infinite virtual environment.

FIG. 1 is a diagram of a system architecture of a three-dimensionalinterface control method according to an embodiment of this disclosure.

A user controls an application based on a three-dimensional userinterface by using a terminal device such as AR or VR glasses. The 3D UIis a medium for direct interaction or information exchange between asystem and the user in a three-dimensional environment, and is anoverall design based on human-machine interaction, operation logic, anda beautiful interface. Although a display range of the virtualenvironment is wider, a distance between the interface and the user islonger. As a result, the user needs to perform an operation in a longdistance and intuitively, and efficiency is reduced. Thethree-dimensional interface control method provided in this embodimentof this disclosure is used to implement division, movement, combination,and control of the three-dimensional interface displayed in a virtualenvironment, to improve operation convenience of the user.

FIG. 2 is a schematic diagram of an embodiment of a separate interfaceaccording to an embodiment of this disclosure. FIG. 2 shows athree-dimensional interface of a drawing application, including threedividable three-dimensional subinterfaces: A, B, and C. A separateinterface refers to dividing different functional areas of a sameapplication in space, and there is usually some gap between differentthree-dimensional interfaces. The separate interface is a common designand layout manner in the VR, AR, or MR three-dimensional interfaces.This design can fully use infinite space in the virtual environment, andhelp the application distinguish different functional areas, such as adisplay area or an operation area, based on an operation habit of theuser. In this embodiment of this disclosure, a functional area that canbe independently displayed on the three-dimensional interface isreferred to as a dividable component.

FIG. 3 is a diagram of an architecture of a three-dimensional interfacecontrol method according to an embodiment of this disclosure.

This method provides a method for dividing a three-dimensional interfacein a virtual environment into three-dimensional subinterfaces. In otherwords, interface division and display 300 are performed, andsubinterfaces obtained by dividing the single three-dimensionalinterface may be separately displayed and controlled, to implementmovement 310 of dividable components or combination 320 of a pluralityof dividable components. In addition, the three-dimensional interfacesof a plurality of applications displayed in the virtual environment maybe divided separately. For the dividable components that have a samefunction in different applications, combination and control 330 of thedividable components of the plurality of applications may be combinedand controlled. For example, controllers of different applications maybe combined into one multi-functional controller, to control theplurality of applications.

The following describes in detail different embodiments.

I. Interface Division and Display

A terminal may divide a first three-dimensional interface intosubinterfaces based on a user operation, and separately display thesubinterfaces in a virtual environment. The user operation includes avoice or a gesture. The following provides specific descriptions.

1. An interface is divided based on a voice of a user. FIG. 4 a is aschematic diagram of an embodiment of a three-dimensional interfacecontrol method according to an embodiment of this disclosure.

411: Determine a first interface based on a first voice of a user.

A terminal determines the first interface based on the first voice thatis made by the user and that is obtained by a voice collection device.

The first interface is an interface of a first application. The firstvoice may be a voice for waking up the first application, for example, avoice such as “music player” or “WeChat”. If the first voice matches avoice in a preset first voice database of the first application, theterminal determines the first interface.

412: If a second voice of the user meets a first preset condition, theterminal determines whether the first interface includes at least twodividable components.

The terminal obtains, by using the voice collection device, the secondvoice made by the user, where the first preset condition may be that thesecond voice matches a voice in a preset second voice database. If thesecond voice matches a voice in the preset second voice database, andthe second voice corresponds to a division instruction, for example, avoice such as “dividing” or “division”, the terminal determines whetherthe first interface includes the at least two dividable components.

413: If the first interface includes the at least two dividablecomponents, the terminal displays, in a virtual environment, a firstsubinterface and a second subinterface that are obtained by dividing thefirst interface.

If the first interface includes the at least two dividable components,the terminal obtains preset division location information on the firstinterface. The first subinterface and the second subinterface that areobtained by dividing the first interface are separately displayed in thevirtual environment based on the division location information. It maybe understood that the first interface may include more than twodividable components. In this step, the terminal may display the morethan two dividable components.

2. An interface is divided based on a gesture of a user.

FIG. 4 b is a schematic diagram of another embodiment of athree-dimensional interface control method according to an embodiment ofthis disclosure.

421: A terminal determines a first interface based on a first gesture ofa user.

The terminal displays a three-dimensional interface of an application ina virtual environment, where the three-dimensional interface includes adividable three-dimensional interface or an indivisiblethree-dimensional interface. The dividable three-dimensional interfaceusually includes a plurality of independent components, for example, adisplay component or a control component. Whether the interface can bedivided may be marked by a developer when a program is launched, and aspecific method is not described herein again.

The terminal determines the first interface based on a first operationof the user.

The first gesture includes a user gesture obtained by the terminal byusing an image collection device. The first gesture may be, for example,a movement of the user gesture. If the first gesture matches a gesturein a preset first gesture database, it means that the first gesture isthe same as the gesture in the preset first gesture database, or adifference between the first gesture and the gesture in the preset firstgesture database is less than a preset threshold. The terminaldetermines the first interface. For example, the first gesture is a usergesture approaching the first interface to a location in which adistance between the user gesture and the first interface is less thanthe preset threshold, or the first gesture is a gesture pointing to thefirst interface.

For example, as shown in FIG. 5 a , the first interface is an interface100 of a video (movie) playback application, and includes two dividablecomponents: a video playback component 120 and a video control component110. The control component is generally a component that operates andcontrols the application, for example, play or pause; or zooming,moving, or rotating performed on a three-dimensional element. Forexample, the video control component 110 shown in FIG. 5 a includesoperation components such as play, pause, and a video progress bar.

422: The terminal determines whether a second operation of the usermeets a preset condition; and if the second operation of the user meetsthe preset condition, performs step 423, or if the second operation ofthe user does not meet the preset condition, performs step 426.

The terminal obtains the second operation performed by the user on thefirst interface by using a sensor system, where the second gesture is auser gesture obtained by the terminal by using the image collectiondevice, for example, a gesture 510 shown in FIG. 5 b.

The first preset condition includes that the second gesture matches agesture in a preset second gesture database. That the terminal maydetermine whether the second gesture matches the gesture in the presetsecond gesture database means that the second gesture is the same as thegesture in the preset second gesture database, or a difference betweenthe second gesture and the gesture in the preset second gesture databaseis less than a preset threshold. The gesture in the second gesturedatabase is a pre-stored “cut gesture”, and may include one or moregestures for indicating interface division. For example, the gesture inthe first gesture database includes a hand state in which five fingersare close together and a palm is flat. If the second operation performedby the user on the first interface meets the first preset condition, theterminal determines whether the first interface is a dividableinterface, and whether the interface is dividable may be marked by adeveloper when a program is launched, and stored in the terminal.

423: If the second gesture meets a second preset condition, the terminaldetermines whether the first interface is a dividable interface.

If the second operation meets the first preset condition, the terminaldetermines whether the first interface is a dividable interface, namely,whether the first interface includes at least two dividable interfaces.The dividable interface includes the at least two dividable components.A division boundary is marked by a developer when a program is launched.The terminal determines whether the first interface is a dividableinterface.

Optionally, an execution occasion of step 423 may be step 421. Afterdetermining the first interface, the terminal determines whether thefirst interface is a dividable interface.

424: If the first interface includes at least two dividable components,the terminal determines location information of a boundary between thedividable components.

If the first interface includes at least two dividable components,namely, if the first interface is a dividable interface, the terminaldetermines the location information of the boundary between thedividable components.

Optionally, the first interface includes two, three, or four dividablecomponents, and a specific quantity is not limited. The locationinformation of the boundary between the dividable components includes aboundary between any two adjacent dividable components on the firstinterface. The boundary between the dividable components may behorizontal, or may be vertical, or both horizontal and vertical, or eveninclude an irregular boundary. This is not specifically limited herein.

Optionally, a division line is displayed at the boundary between thedividable components of the first interface, where the division lineincludes the boundary between any two dividable components on the firstinterface. For example, FIG. 5 c shows a division line 700.

Optionally, the dividable component is highlighted to indicate that thisarea is dividable.

Optionally, a feedback box is displayed at the boundary of the dividablecomponents. The feedback box is a solid line displayed at an interfacecontour, and is used to limit a size of the dividable interface. Forexample, FIG. 5 b shows a feedback box 920 of the video playbackcomponent 120 and a feedback box 910 of the video control component 110.

425: If the first interface includes at least two dividable components,and the second gesture meets the second preset condition, the terminalseparately displays a first subinterface and a second subinterface thatare obtained by dividing the first interface.

The second preset condition includes that the second gesture moves alongthe division line, or a distance between the second gesture and thedivision line is less than a preset threshold and the second gesture isparallel to the division line. It may be understood that along andparallel herein may include a specific error. For example, the secondgesture includes a gesture of moving a flat palm with fingers togetherby a preset distance in a direction in which a palm plane extends, wherethe moving direction of the gesture is parallel to the boundary betweenthe dividable components of the first interface. It should be notedthat, based on different location information of the boundary betweenthe dividable components, a movement track that needs to be satisfied bya gesture is also different. If the location information of the boundaryis horizontal, the movement track of the gesture is also horizontal; ifthe location information of the boundary is vertical, the movement trackof the gesture is also vertical; and if the location information of theboundary is an irregular curve, the movement track of the gesture alsoneeds to match the irregular curve that may be similar within a certainerror range.

Optionally, the terminal detects a gesture movement distance of theuser, and highlights a cut line when determining that the gesturemovement direction of the user is parallel to the cut line and thedistance is less than a preset threshold. Optionally, as the gesturemovement distance changes, the cut line may feed back the gesturemovement distance by using a color change, to indicate a current cuttingprogress.

For example, as shown in FIG. 5 d , a user gesture moves from a location511 to a location 512, and a color of a cut part 710 of a cut linechanges (in the figure, a dashed line indicating an uncut part 720 ofthe cut line is a densely dashed line, a dashed line indicating the cutpart 710 of the cut line is a sparsely dashed line, and different dashedlines are used to represent a color change of the cut part 710 of thecut line).

If the first interface is a dividable interface and the second operationmeets the second preset condition, the terminal displays, in the virtualenvironment, the first subinterface and the second subinterface that areobtained by dividing the first interface.

For example, as shown in FIG. 5 e , after the terminal determines thatthe user keeps a cut gesture in the air from one end point (namely, thelocation 511) of the division line to another end point (namely, thelocation 513), the system determines that the cutting action iscompleted.

As shown in FIG. 5 f to FIG. 5 h , there are multiple manners in whichthe terminal displays, in the virtual environment, the firstsubinterface and the second subinterface that are obtained by dividingthe first interface.

A before-division initial location of the first subinterface is a firstinitial location, and an after-division location is a first location. Abefore-division initial location of the second subinterface is a secondinitial location, and an after-division location is a second location.

Optionally, the first location is the same as the first initiallocation, and a distance between the second location and the secondinitial location in a direction away from the first initial location isa first preset value. For example, as shown in FIG. 5 f , the firstsubinterface is the video playback component 120, and the secondsubinterface is the video control component 110. A location of the videoplayback component 120 remains unchanged, and the video controlcomponent 110 is moved downward by a preset distance from a location 111to a location 112. In this case, a video playback area and a videocontrol area are two independent display interfaces.

Optionally, the second location is the same as the second initiallocation, and a distance between the first location and the firstinitial location in a direction away from the second initial location isa second preset value. For example, as shown in FIG. 5 g , a location ofthe video control component 110 remains unchanged, and the videoplayback component 120 is moved upward by a preset distance from alocation 121 to a location 122. Therefore, the video playback component120 and the video control component 110 are spaced and displayed as twoindependent interfaces.

Optionally, a distance between the first location and the first initiallocation in a direction away from the second initial location is a thirdpreset value, and a distance between the second location and the secondinitial location in a direction away from the first initial location isa fourth preset value. For example, as shown in FIG. 5 h , the videoplayback component 120 is moved upward by a preset distance from thelocation 121 to the location 122, and the video control component 110 ismoved downward by a preset distance from the location 111 to thelocation 112. Therefore, the video playback component 120 and the videocontrol component 110 are spaced and displayed as two independentinterfaces.

Optionally, when the first interface is divided into at least twosubinterfaces, there are multiple layout manners, and the layout mannersmay be a combination of one or more of moving up and down, moving leftand right, or moving forward and backward. This is not specificallylimited herein.

Optionally, location information of the first subinterface on the firstinterface and location information of the second subinterface on thefirst interface during cutting are recorded. The location informationis, for example, coordinate data, and may be used to determine arelative location between the first subinterface and the secondsubinterface when the first subinterface and the second subinterface aresubsequently combined.

The first subinterface and the second subinterface that are obtained bydividing the first interface may be separately displayed and controlled,and the control includes adjusting an interface size, moving, rotating,combining, operating a functional component on the interface, and thelike. This is not specifically limited herein.

II. Dividable Component Movement

The following describes a subinterface moving method. FIG. 6 is aschematic diagram of an embodiment of a subinterface control methodaccording to an embodiment of this disclosure.

601: A terminal determines, based on a third operation of a user, afirst subinterface located in a third location.

The terminal may determine the first subinterface based on the thirdoperation of the user, where a location of the first subinterface is thethird location.

Optionally, if a distance between a third gesture operation and thefirst subinterface is less than the preset threshold, the firstsubinterface is determined.

Optionally, after determining the first subinterface, the terminal maydisplay a feedback identifier. As shown in FIG. 7 a , the feedbackidentifier appears on a contour of a dividable component. The feedbackidentifier includes a feedback box for indicating a size of a selectedinterface or a highlighting component for feeding back to the user thatthe interface is selected.

As shown in FIG. 7 a , a distance between a location 514 of a usergesture and the first subinterface is less than the preset threshold.Therefore, the video control component 110 is determined, and a feedbackidentifier 910 appears on a contour of the video control component 110.

602: The terminal moves the first subinterface based on a fourthoperation of the user.

Optionally, the fourth gesture includes keeping a “pinch grip” gestureand moving, where the “pinch grip” gesture is a gesture in which a thumband an index finger change from a separated state to a touch state, andthe “pinch grip” gesture requires that a distance between the usergesture and the first subinterface be less than or equal to the presetthreshold.

Optionally, the fourth gesture includes keeping an “open palm” gestureand moving. The “open palm” gesture is a hand state in which fivefingers are close together and a palm is upward. It should be notedthat, in a moving process of the “open palm” gesture, a distance betweenthe user gesture and the first subinterface is not limited, and thefirst subinterface follows the user gesture.

The terminal determines the first subinterface selected by the user byusing the “pinch grip” gesture, and records a relative displacement of ahand (for example, an initial location of the hand is (x1, y1, z1), afirst location is (x2, y2, z2), and a relationship between the initiallocation and the first location is used to describe the relativedisplacement, for example, a 1 cm movement in a forward direction of thex-axis, a 1 cm movement in a forward direction of the y-axis, and a 1 cmmovement in a forward direction of the z-axis). The location of thecomponent is displayed by the relative displacement based on a presetproportion relationship, and a display effect is that the firstsubinterface moves with a change of the location of the gesture.

603: The terminal displays the first subinterface located in a fourthlocation.

The terminal stops moving the first subinterface based on a fifthoperation of the user, and the first subinterface stops moving. When thefifth operation is performed, a location of the first interface is thefourth location.

The terminal stops moving the first subinterface based on the fifthoperation of the user. Optionally, the fifth operation includescanceling the fourth operation, and may be specifically that a gestureof the fourth operation changes, or a distance between the user gestureand the moving interface is greater than or equal to the presetthreshold. This is not specifically limited herein.

Optionally, if the first subinterface is moved by using the “pinch grip”gesture, and the “pinch grip” gesture is canceled, the firstsubinterface may stop moving, and the first subinterface stays in alocation of the first subinterface when the “pinch grip” gesture iscanceled on the terminal.

Optionally, if the first subinterface is moved by using the “open palm”gesture, and the “open palm” gesture is canceled, the first subinterfacemay stop moving, and the first subinterface stays in a location of thefirst subinterface when the “open palm” gesture is canceled on theterminal.

Example 1: Refer to FIG. 7 b and FIG. 7 c . As shown in FIG. 7 b , the“pinch grip” gesture moves from the location 514 to the location 515,and the video control component 110 accordingly moves from a location113 to a location 114. If the user cancels the “pinch grip” gesture, thevideo control component 110 stops moving. As shown in FIG. 7 c , thevideo control component 110 stays in the location 114.

Example 2: Refer to FIG. 7 d and FIG. 7 e . Considering that in someapplication scenarios, the user needs to move the interface by a largedistance, it is difficult to perform an operation of keeping the “pinchgrip” gesture and moving. As shown in FIG. 7 d , the interface mayfollow an “open palm” gesture 530. In a movement process, a distancebetween the user gesture and the interface is not limited, and theinterface moves along with the “open palm” gesture. As shown in FIG. 7 e, the video control component 110 moves from the location 113 to alocation 115.

FIG. 8 is a schematic diagram of an embodiment of combination of aplurality of subinterfaces according to an embodiment of thisdisclosure.

801: A terminal moves a first subinterface and/or a second subinterface.

The terminal moves the first subinterface or the second subinterface.

Optionally, the terminal simultaneously moves the first subinterface andthe second subinterface.

802: If a distance between the first subinterface and the secondsubinterface is less than or equal to a threshold, the terminal combinesthe first subinterface and the second subinterface into a secondinterface.

For example, when a user simultaneously “pinches” and moves the firstsubinterface and the second subinterface by two hands on the terminal,and it is determined that the distance between the first subinterfaceand the second subinterface is less than or equal to a preset threshold,the terminal combines the first subinterface and the second subinterfaceinto the second interface. Optionally, a layout of the second interfacemay be the same as or may be different from a layout of the firstinterface. The second interface returns based on before-divisionlocation information, for example, coordinate values, that is recordedby the terminal and that is of the first subinterface and the secondsubinterface, so that the combined second interface is the same as thebefore-division first interface.

Optionally, the relative location of the first subinterface and thesecond subinterface on the second interface is determined based on thelocation information of the first subinterface and the secondsubinterface on the first interface when the first interface is divided.

Optionally, if the distance between the first subinterface and thesecond subinterface is less than or equal to the preset threshold, abounding box is displayed at an interface contour, or both the firstsubinterface and the second subinterface are highlighted.

For example, as shown in FIG. 9 a , the terminal determines the videocontrol component 110 based on a first “pinch grip” gesture 550, anddetermines the video playback component 120 based on a second “pinchgrip” gesture 540. The video control component 110 and the videoplayback component 120 are moved to approach each other by using a“pinch grip” gesture.

As shown in FIG. 9 b , when a distance between the video controlcomponent 110 and the video playback component 120 is less than or equalto a preset threshold, the first subinterface and the secondsubinterface are combined into the second interface for display.Optionally, although not shown in the figure, when the distance betweenthe video control component 110 and the video playback component 120 isless than or equal to the preset threshold, the boundary box mayalternatively be displayed at contours of the first subinterface and thesecond subinterface to feed back information that the user interfacesare to be combined.

III. Combination and Control of Dividable Components Among a Pluralityof Applications

FIG. 10 is a schematic diagram of a combination method of subinterfacesof a plurality of applications according to an embodiment of thisdisclosure.

1001: A terminal displays subinterfaces of a plurality of applicationsin a virtual environment.

The terminal may display interfaces of two or more applications,including a first subinterface and a second subinterface that areobtained by dividing a first interface of a first application, and athird subinterface and a fourth subinterface that are obtained bydividing a second interface of a second application.

For example, as shown in FIG. 11 a , the terminal displays a videoplayer display interface and a music playback interface that areseparated. The video player display interface is divided into a videocontrol component 110 and a video playback component 120, and the musicplay interface is divided into a music control component 210 and a musicplay component 220. Specifically, the video playback component 120includes information of video playback content, and the music playcomponent 220 includes information of music play content. The videocontrol component 110 includes play, pause, a progress bar, and abrightness component, and the music control component 210 includes play,pause, and a progress bar component.

1002: A terminal moves the first subinterface or the third subinterface.

The terminal may move the first subinterface and the third subinterfacebased on a user operation. Optionally, the terminal simultaneously movesthe first subinterface and the third subinterface.

For example, as shown in FIG. 11 b , the system detects and identifiesthe first “pinch grip” gesture 550, and determines the video playbackcontroller 110, where the video playback controller 110 moves along withthe first “pinch grip” gesture 550. In addition, a second gesture 560determines the music play controller 210, where the music playcontroller 210 moves along with the second gesture 560. Optionally, afeedback identifier is displayed at contours of the video playbackcontroller 110 and the music play controller 210, and may be a feedbackbox or a highlighting component.

1003: If a distance between the first subinterface and the thirdsubinterface is less than or equal to a threshold, the terminal combinesthe first subinterface and the third subinterface for display.

If the distance between the first subinterface and the thirdsubinterface is less than or equal to a preset threshold, and the firstsubinterface and the third subinterface meet a preset condition, thefirst subinterface and the third subinterface are combined into a fifthsubinterface, and the fifth subinterface is displayed.

The preset condition includes that the first subinterface and the thirdsubinterface include components of a same type.

For example, as shown in FIG. 11 c , the first subinterface is the videoplayback controller 110, and the third subinterface is the music playcontroller 210. The two controllers are combined into the fifthsubinterface, namely, a multi-functional controller 300, by moving thetwo subinterfaces close to each other. A combined display interface isshown in FIG. 11 d.

The system determines that the user simultaneously releases the pinchgrip of both hands and combines the two controllers. Functions of thetwo controllers to be combined are determined. If the functions are thesame, one of the functions is reserved. If the functions are different,the functions are combined. In this embodiment, functions of the videoplayback controller and the music play controller are consistent,including functions such as play, previous, next, and a progress bar.After the combination is completed, one of the controllers may bereserved as a combined control for multi-task control. If functions ofthe two controllers are inconsistent, for example, a video player and athree-dimensional modeling controller (rotating, zooming, rotating, andthe like), all functional components in the two controllers arededuplicated and combined into a new combined control.

The terminal may control the combined fifth subinterface based on a useroperation. Optionally, the terminal may further perform combined displayon the first subinterface and the third subinterface based on a voice ofthe user. For example, when the user separately “pinches” the videoplayback controller and the music play controller, and sends a voiceinstruction of “To combine controllers”, the video playback controllerand the music play controller may be combined for display.

1004: The terminal displays an indication identifier based on a useroperation.

Step 340 is performed after the control combination in 330 is completed.

Because the fifth subinterface may be configured to control the firstapplication and the second application, to distinguish control objects,after determining the fifth subinterface based on the user operation,the terminal may display the indication identifier, where the indicationidentifier is used to determine an interface currently associated withthe fifth subinterface. Optionally, the indication identifier isdisplayed at an edge of the fifth subinterface. Optionally, theindication identifier is an arrow displayed above the fifthsubinterface, and the arrow is perpendicular to an upper boundary of thefifth subinterface and is in a direction away from the fifthsubinterface.

Optionally, the fifth subinterface is determined based on the “pinchgrip” gesture of the user on the fifth subinterface, and the indicationidentifier is displayed.

For example, as shown in FIG. 11 e , based on the “pinch grip” gestureof the multi-functional controller 300, the indication identifier,namely, an arrow 310, is displayed in a vertical upper boundary and inan outward direction at a central location of an upper boundary of themulti-functional controller 300, where a direction indicated by thearrow 310 is used to determine an interface currently controlled by themulti-functional controller 300.

1005: If the indication identifier points to the second subinterface,the terminal controls the first application based on a user operationperformed on the fifth subinterface.

The terminal may move the fifth subinterface based on the useroperation, and an indication identifier of the fifth subinterface movesalong with the fifth subinterface. If the indication identifier pointsto the second subinterface, the terminal determines that a controlobject of the fifth subinterface is the first application correspondingto the second subinterface.

For example, as shown in FIG. 11 f , the multi-functional controller 300is moved based on the gesture 570. If the arrow 310 aims at the videoplayback component 120, specifically if an extension line of the arrowoverlaps with the video playback component 120, the multi-functionalcontroller 300 is configured to control an application, namely, a videoapplication, corresponding to the video playback component 120. As shownin FIG. 11 g , the video application is controlled by using a controlcomponent in the multi-functional controller 300.

Optionally, an aimed video playback task interface displaysidentification feedback, where the identification feedback includes afeedback box, and indicates an area in which the second subinterface iscontrolled; or the identification feedback is highlighting the secondsubinterface, or the like.

1006: If the indication identifier points to the fourth subinterface,the terminal controls the second application based on the user operationperformed on the fifth subinterface.

Similarly, if the indication identifier points to the fourthsubinterface, the terminal determines that the control object of thefifth subinterface is the second application corresponding to the fourthsubinterface.

For example, as shown in FIG. 11 h , the multi-functional controller 300is moved based on the gesture 570. If the arrow 310 aims at the musicplay component 220, the multi-functional controller 300 is configured tocontrol an application, namely, a music application, corresponding tothe music play component 220. It may be understood that, when there aremore than two control components that can be combined, a selectioninstruction, for example, “To combine controllers of video and music”,for a control component that needs to be combined in the voiceinstruction may be further identified.

Optionally, the fifth subinterface may be further controlled in a voicecontrol manner, and the voice instruction needs to include a specificcontrol action (such as play or pause) and a target application (such asa video application) to be controlled. For example, if the user sends aninstruction of “To play a video”, steps of alignment and clicking on avirtual interface may be omitted. Because a main purpose of step 340 andstep 350 is to select a target control component and a control function,a line-of-sight aiming manner or a brain wave detection manner may alsobe used in addition to a voice manner. For example, if the user firstlooks at a video playback interface, and then looks at a play button ofthe control component, the user may perform a play operation on thevideo playback interface.

The foregoing describes the interface control method provided in thisdisclosure, and the following describes a terminal that implements theinterface control method. FIG. 12 is a schematic diagram of anembodiment of a terminal according to an embodiment of this disclosure.

Only one or more of the modules in FIG. 12 may be implemented by usingsoftware, hardware, firmware, or a combination thereof. The software orfirmware includes but is not limited to computer program instructions orcode, and may be executed by a hardware processor. The hardware includesbut is not limited to various integrated circuits, for example, acentral processing unit (CPU), a digital signal processor (DSP), a fieldprogrammable gate array (FPGA), or an application-specific integratedcircuit (ASIC).

The terminal includes:

an obtaining unit 1201, configured to obtain a first operation on afirst interface displayed in a virtual environment, where the firstinterface includes at least two dividable components; and

a display unit 1202, configured to display, in the virtual environmentbased on division location information in response to the firstoperation, a first subinterface and a second subinterface that areobtained by dividing the first interface, where the first subinterfaceincludes a first dividable component of the at least two dividablecomponents, the second subinterface includes a second dividablecomponent adjacent to the first dividable component, and the divisionlocation information includes a boundary between the first dividablecomponent and the second dividable component.

Optionally, the first operation includes first voice informationobtained by the terminal by using a voice collection apparatus, or afirst gesture obtained by the terminal by using a sensor system.

Optionally, the first gesture includes: moving a flat palm with fingerstogether by a preset distance in a first direction in which a palm planeextends, where the first direction in which the palm plane extends isparallel to the boundary.

Optionally, the display unit 1202 is further configured to display adivision line on a boundary between the first dividable component andthe second dividable component.

Optionally, the display unit 1202 is specifically configured to:

display the first subinterface in a first location, and display thesecond subinterface in a second location.

The first location is the same as a first initial location, a distancebetween the second location and a second initial location in a directionaway from the first initial location is a first preset value, the firstinitial location is a location of the first subinterface on the firstinterface when the terminal obtains a first instruction, and the secondinitial location is a location of the second subinterface on the firstinterface when the terminal obtains the first instruction; or

the second location is the same as a second initial location, and adistance between the first location and a first initial location in adirection away from the second initial location is a second presetvalue; or

a distance between the first location and a first initial location in adirection away from a second initial location is a third preset value,and a distance between the second location and the second initiallocation in a direction away from the first initial location is a fourthpreset value.

Optionally, the terminal further includes:

a processing unit 1203, configured to move the first subinterface inresponse to a second operation on the first subinterface.

Optionally, the second operation includes: a pinch grip gesture pointingto the first subinterface, where the pinch grip gesture includes a handstate in which a thumb approaches at least one of four fingers, andkeeping the pinch grip gesture and moving; or

touching and holding the first subinterface and moving; or

keeping an open palm gesture and moving, where the open palm gestureincludes a hand state with fingers together and a palm up.

Optionally, the terminal further includes:

a processing unit 1203, configured to move the first subinterface.

The display unit 1202 is further configured to: if a distance betweenthe first subinterface and the second subinterface is less than a firstthreshold, display, in the virtual environment, a second interfaceobtained by combining the first subinterface and the secondsubinterface, where a layout of the second interface is the same as ordifferent from a layout of the first interface.

Optionally, the terminal further includes:

The display unit 1202 is further configured to display a third interfacein the virtual environment, where the first interface is an interface ofa first application, and the third interface is an interface of a secondapplication.

The processing unit 1203 is further configured to move the firstsubinterface and/or the third interface.

The display unit 1202 is further configured to: if a distance betweenthe first subinterface and the third interface is less than a secondthreshold, display, in the virtual environment, a fourth interfaceobtained by combining the first subinterface and the third interface.

Optionally, the display unit 1202 is further configured to display athird subinterface and a fourth subinterface in the virtual environment,where the third subinterface and the fourth subinterface are obtained bydividing the third interface, the first interface is the interface ofthe first application, and the third interface is the interface of thesecond application.

The processing unit 1203 is further configured to move the firstsubinterface and/or the third subinterface.

The display unit 1202 is further configured to: if a distance betweenthe first subinterface and the third subinterface is less than a thirdthreshold, display, in the virtual environment, a fifth subinterfaceobtained by combining the first subinterface and the third subinterface.

Optionally, the first subinterface includes a first icon for controllinga first function of the first application. The third subinterfaceincludes a second icon for controlling a second function of the secondapplication. The first function is the same as the second function.

The fifth subinterface includes a third icon. The third icon is used tocontrol the first function of the first application and the secondfunction of the second application.

Optionally, the first subinterface is an input interface of the firstapplication. The second subinterface is an output interface of the firstapplication. The third subinterface is an input interface of the secondapplication. The fourth subinterface is an output interface of thesecond application.

The display unit 1202 is further configured to display an indicationidentifier on an edge of the fifth subinterface, where the indicationidentifier is used to determine an application currently controlled bythe fifth subinterface.

The terminal further includes:

a control unit 1204, configured to: when the indication identifierpoints to the second subinterface, control the first application basedon an input operation performed on the fifth subinterface.

The control unit 1204 is further configured to: when the indicationidentifier points to the fourth subinterface, control, by the terminal,the second application based on an input operation performed on thefifth subinterface.

FIG. 13 is a schematic diagram of another embodiment of a terminalaccording to an embodiment of this disclosure.

For ease of understanding, the following describes, by using an example,a structure of a terminal 100 provided in this embodiment of thisdisclosure. FIG. 13 is a schematic diagram of a structure of a terminalaccording to an embodiment of this disclosure.

As shown in FIG. 13 , the terminal 100 may include a processor 110, anexternal memory interface 120, an internal memory 121, a universalserial bus (USB) port 130, a charging management module 140, a powermanagement module 141, a battery 142, an antenna 1, an antenna 2, amobile communication module 150, a wireless communication module 160, anaudio module 170, a speaker 170A, a receiver 170B, a microphone 170C, aheadset jack 170D, a sensor module 180, a button 190, a motor 191, anindicator 192, a camera 193, a display 194, a subscriber identity module(SIM) card interface 195, and the like. The sensor module 180 mayinclude a pressure sensor 180A, a gyro sensor 180B, a barometricpressure sensor 180C, a magnetic sensor 180D, an acceleration sensor180E, a distance sensor 180F, a proximity sensor 180G, a fingerprintsensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambientlight sensor 180L, a bone conduction sensor 180M, and the like.

It may be understood that the structure shown in this embodiment of thisdisclosure does not constitute a specific limitation on the terminal100. In some other embodiments of this disclosure, the terminal 100 mayinclude more or fewer components than those shown in the figure, or somecomponents may be combined, or some components may be split, or theremay be a different component layout. The components shown in the figuremay be implemented by hardware, software, or a combination of softwareand hardware.

The processor 110 may include one or more processing units. For example,the processor 110 may include an application processor (AP), a modemprocessor, a graphics processing unit (GPU), an image signal processor(ISP), a controller, a memory, a video codec, a digital signal processor(DSP), a baseband processor, a neural-network processing unit (NPU),and/or the like. Different processing units may be independentcomponents, or may be integrated into one or more processors.

The controller may be a nerve center and a command center of theterminal 100. The controller may generate an operation control signalbased on instruction operation code and a time sequence signal, tocomplete control of instruction reading and instruction execution.

A memory may be further disposed in the processor 110, and is configuredto store instructions and data. In some embodiments, the memory in theprocessor 110 is a cache memory. The memory may store instructions ordata that has been used or is cyclically used by the processor 110. Ifthe processor 110 needs to use the instructions or the data again, theprocessor may directly invoke the instructions or the data from thememory. This avoids repeated access, reduces waiting time of theprocessor 110, and improves system efficiency.

In some embodiments, the processor 110 may include one or moreinterfaces. The interface may include an inter-integrated circuit (IIC)interface, an inter-integrated circuit sound (IICS) interface, a pulsecode modulation (PCM) interface, a universal asynchronousreceiver/transmitter (UART) interface, a mobile industry processorinterface (MIPI), a general-purpose input/output (GPIO) interface, asubscriber identity module (SIM) interface, a universal serial bus (USB)port, and/or the like.

It may be understood that an interface connection relationship betweenthe modules in this embodiment of this disclosure is merely an examplefor description, and does not constitute a limitation on the structureof the terminal 100. In some other embodiments of this disclosure, theterminal 100 may alternatively use an interface connection mannerdifferent from that in the foregoing embodiment, or a combination of aplurality of interface connection manners.

The charging management module 140 is configured to receive a charginginput from the charger. The charger may be a wireless charger or a wiredcharger. In some embodiments of wired charging, the charging managementmodule 140 may receive a charging input of a wired charger through theUSB port 130.

The power management module 141 is configured to connect the battery 142and the charging management module 140 to the processor 110. The powermanagement module 141 receives input of the battery 142 and/or thecharging management module 140, to supply power to the processor 110,the internal memory 121, an external memory, the display 194, the camera193, the wireless communication module 160, and the like.

A wireless communication function of the terminal 100 may be implementedthrough the antenna 1, the antenna 2, the mobile communication module150, the wireless communication module 160, the modem processor, thebaseband processor, and the like.

In some feasible implementations, the terminal 100 may communicate withanother device by using a wireless communication function. For example,the terminal 100 may communicate with a second electronic device 200,the terminal 100 establishes a projection connection to the secondelectronic device 200, and the terminal 100 outputs projection data tothe second electronic device 200. The projection data output by theterminal 100 may be audio and video data.

The antenna 1 and the antenna 2 are configured to transmit and receivean electromagnetic wave signal. Each antenna in the terminal 100 may beconfigured to cover one or more communication frequency bands. Differentantennas may be further multiplexed, to improve antenna utilization. Forexample, the antenna 1 may be multiplexed as a diversity antenna of awireless local area network. In some other embodiments, the antenna maybe used in combination with a tuning switch.

The mobile communication module 150 may provide a solution that isapplied to the terminal 100 and that includes wireless communicationsuch as 1G, 3G, 4G, 5G, and the like. The mobile communication module150 may include at least one filter, a switch, a power amplifier, a lownoise amplifier (LNA), and the like. The mobile communication module 150may receive an electromagnetic wave through the antenna 1, performprocessing such as filtering or amplification on the receivedelectromagnetic wave, and transmit the electromagnetic wave to the modemprocessor for demodulation. The mobile communication module 150 mayfurther amplify a signal modulated by the modem processor, and convertthe signal into an electromagnetic wave for radiation through theantenna 2. In some embodiments, at least some functional modules in themobile communication module 150 may be disposed in the processor 110. Insome embodiments, at least some functional modules of the mobilecommunication module 150 may be disposed in a same device as at leastsome modules of the processor 110.

The modem processor may include a modulator and a demodulator. Themodulator is configured to modulate a to-be-sent low-frequency basebandsignal into a medium-high frequency signal. The demodulator isconfigured to demodulate a received electromagnetic wave signal into alow-frequency baseband signal. Then, the demodulator transmits thelow-frequency baseband signal obtained through demodulation to thebaseband processor for processing. The low-frequency baseband signal isprocessed by the baseband processor and then transmitted to theapplication processor. The application processor outputs a sound signalby an audio device (which is not limited to the speaker 170A, thereceiver 170B, or the like), or displays an image or a video by thedisplay 194. In some embodiments, the modem processor may be anindependent component. In some other embodiments, the modem processormay be independent of the processor 110, and is disposed in a samedevice as the mobile communication module 150 or another functionalmodule.

The wireless communication module 160 may provide a wirelesscommunication solution that is applied to the terminal 100, and thatincludes a wireless local area network (WLAN) (for example, a wirelessfidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellitesystem (GNSS), frequency modulation (FM), a near field communication(NFC) technology, an infrared (IR) technology, and the like. Thewireless communication module 160 may be one or more componentsintegrating at least one communication processor module. The wirelesscommunication module 160 receives an electromagnetic wave by the antenna1, performs frequency modulation and filtering processing on anelectromagnetic wave signal, and sends a processed signal to theprocessor 110. The wireless communication module 160 may further receivea to-be-sent signal from the processor 110, perform frequency modulationand amplification on the signal, and convert the signal into anelectromagnetic wave for radiation through the antenna 2.

In some embodiments, the antenna 1 of the terminal 100 is coupled to themobile communication module 150, and the antenna 2 is coupled to thewireless communication module 160, so that the terminal 100 maycommunicate with a network and another device by using a wirelesscommunication technology. The wireless communication technology mayinclude a global system for mobile communications (GSM), a generalpacket radio service (GPRS), code division multiple access (CDMA),wideband code division multiple access (WCDMA), time-division codedivision multiple access (TD-SCDMA), long term evolution (LTE), BT, aGNSS, a WLAN, NFC, FM, an IR technology, and/or the like. The GNSS mayinclude a global positioning system (GPS), a global navigation satellitesystem (GLONASS), a BeiDou navigation satellite system (BDS), aquasi-zenith satellite system (QZSS), and/or a satellite basedaugmentation system (SBAS).

The terminal 100 may implement a display function by using the GPU, thedisplay 194, the application processor, and the like. The GPU is amicroprocessor for image processing, and is connected to the display 194and the application processor. The GPU is configured to: performmathematical and geometric computation, and render an image. Theprocessor 110 may include one or more GPUs, which execute programinstructions to generate or change display information.

The display 194 is configured to display an image, a video, and thelike. The display 194 includes a display panel. The display panel may bea liquid crystal display (LCD), an organic light-emitting diode (OLED),an active-matrix organic light emitting diode (AMOLED), a flexiblelight-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, aquantum dot light emitting diode (QLED), or the like. In someembodiments, the terminal 100 may include one or N displays 194, where Nis a positive integer greater than 1.

In some feasible implementations, the display 194 may be configured todisplay interfaces for output by a system of the terminal 100. For allthe interfaces output by the terminal 100, refer to related descriptionsof the subsequent embodiments.

The terminal 100 may implement an image shooting function by using theISP, the camera 193, the video codec, the GPU, the display 194, theapplication processor, and the like.

The ISP is configured to process data fed back by the camera 193. Forexample, during photographing, a shutter is pressed, and light istransmitted to a photosensitive element of the camera through a lens. Anoptical signal is converted into an electrical signal, and thephotosensitive element of the camera transmits the electrical signal tothe ISP for processing, to convert the electrical signal into a visibleimage. The ISP may further perform algorithm optimization on noise,brightness, and complexion of the image. The ISP may further optimizeparameters such as exposure and a color temperature of a photographingscenario. In some embodiments, the ISP may be disposed in the camera193.

The camera 193 is configured to capture a static image or a video. Anoptical image of an object is generated through the lens, and isprojected onto the photosensitive element. The photosensitive elementmay be a charge coupled device (CCD) or a complementarymetal-oxide-semiconductor (CMOS) phototransistor. The light-sensitiveelement converts an optical signal into an electrical signal, and thentransmits the electrical signal to the ISP to convert the electricalsignal into a digital image signal. The ISP outputs the digital imagesignal to the DSP for processing. The DSP converts the digital imagesignal into an image signal in a standard format such as RGB or YUV. Insome embodiments, the terminal 100 may include one or N cameras 193,where N is a positive integer greater than 1.

The digital signal processor is configured to process a digital signal,and may process another digital signal in addition to the digital imagesignal.

The video codec is configured to compress or decompress a digital video.The terminal 100 may support one or more video codecs. In this way, theterminal 100 may play or record videos in a plurality of encodingformats, for example, moving picture experts group (MPEG)-1, MPEG-2,MPEG-3, and MPEG-4.

The NPU is a neural-network (NN) computing processor. The NPU quicklyprocesses input information by drawing on a structure of a biologicalneural network, for example, by drawing on a transfer mode between humanbrain neurons, and may further continuously perform self-learning.Applications such as intelligent cognition of the terminal 100 may beimplemented through the NPU, for example, image recognition, facialrecognition, speech recognition, and text understanding

The external memory interface 120 may be used to connect to an externalmemory card, for example, a micro SD card, to extend a storagecapability of the terminal 100. The external memory card communicateswith the processor 110 through the external memory interface 120, toimplement a data storage function. For example, files such as music andvideos are stored in the external storage card.

The internal memory 121 may be configured to store computer-executableprogram code. The executable program code includes instructions. Theprocessor 110 runs the instructions stored in the internal memory 121,to implement various function applications and data processing of theterminal 100. The internal memory 121 may include a program storage areaand a data storage area. The program storage area may store an operatingsystem, an application required by at least one function (for example, avoice playing function or an image playing function), and the like. Thedata storage area may store data (for example, audio data and a phonebook) and the like created when the terminal 100 is used. In addition,the internal memory 121 may include a high-speed random access memory,or may include a nonvolatile memory, for example, at least one magneticdisk storage device, a flash memory, or a universal flash storage (UFS).

The terminal 100 may implement audio functions such as music playing andrecording by using the audio module 170, the speaker 170A, the receiver170B, the microphone 170C, the headset jack 170D, the applicationprocessor, and the like. In some feasible implementations, the audiomodule 170 may be configured to play a sound corresponding to a video.For example, when the display 194 displays a video play picture, theaudio module 170 outputs a video play sound.

The audio module 170 is configured to convert digital audio informationinto an analog audio signal for output, and is also configured toconvert an analog audio input into a digital audio signal.

The speaker 170A, also referred to as a “loudspeaker”, is configured toconvert an audio electrical signal into a sound signal.

The receiver 170B, also referred to as an “earpiece”, is configured toconvert an audio electrical signal into a sound signal.

The microphone 170C, also referred to as a “mike” or a “mic”, isconfigured to convert a sound signal into an electrical signal.

The headset jack 170D is configured to connect to a wired headset. Theheadset jack 170D may be a USB port 130, or may be a 3.5 mm open mobileterminal platform (OMTP) standard interface or cellulartelecommunications industry association of the USA (CTIA) standardinterface.

The pressure sensor 180A is configured to sense a pressure signal, andcan convert the pressure signal into an electrical signal. In someembodiments, the pressure sensor 180A may be disposed on the display194. The gyro sensor 180B may be configured to determine a motionposture of the terminal 100. The barometric pressure sensor 180C isconfigured to measure barometric pressure.

The acceleration sensor 180E may detect magnitudes of accelerations ofthe terminal 100 in various directions (usually on three axes or sixaxes). A magnitude and a direction of gravity may be detected when theterminal 100 is still. The acceleration sensor 180E may be furtherconfigured to identify a posture of the terminal, and is applied to anapplication such as switching between a landscape mode and a portraitmode or a pedometer.

The distance sensor 180F is configured to measure a distance.

The ambient light sensor 180L is configured to sense ambient lightbrightness.

The fingerprint sensor 180H is configured to collect a fingerprint.

The temperature sensor 180J is configured to detect a temperature.

The touch sensor 180K is also referred to as a “touch panel”. The touchsensor 180K may be disposed on the display 194. The touch sensor 180Kand the display 194 form a touchscreen that is also referred to as a“touch screen”. The touch sensor 180K is configured to detect a touchoperation performed on or near the touch sensor. The touch sensor maytransfer the detected touch operation to the application processor todetermine a type of the touch event. A visual output related to thetouch operation may be provided through the display 194. In some otherembodiments, the touch sensor 180K may alternatively be disposed on asurface of the terminal 100 at a position different from that of thedisplay 194.

The button 190 includes a power button, a volume button, and the like.The button 190 may be a mechanical button, or a touch button. Theterminal 100 may receive a button input, and generate a button signalinput related to a user setting and function control of the terminal100.

The motor 191 may generate a vibration prompt.

The indicator 192 may be an indicator light, and may be configured toindicate a charging status and a power change, or may be configured toindicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is configured to connect to a SIM card.

FIG. 14 is a schematic diagram of an HMD device according to anembodiment of this disclosure.

An HMD device in this disclosure is shown in FIG. 14 . The HMD deviceincludes:

A processor 1401, configured to: generate a corresponding operationcontrol signal, send the operation control signal to a correspondingpart of the device, and read and process data in software, especiallyread and process data and a program in the memory, so that eachfunctional module in the device performs a corresponding function, tocontrol a corresponding part to perform an action based on aninstruction requirement. For example, the processor 1401 is applied to avariety of media processing algorithms, including man-machineinteraction, motion tracking/prediction, rendering display, audioprocessing, and the like.

A sensor system 1402, configured to collect, obtain, or sendinformation, including image information and distance information, suchas user gesture information in this disclosure. The sensor system inthis disclosure may include a 3-axis or 6-axis sensor, and is configuredto: obtain motion information of the HMD device, for example, an angularvelocity and a linear acceleration; position, track, and recognize ahand motion; and present the recognized hand motion in a display of theVR device. The sensor system further obtains static and dynamic featuresof a hand. Static characteristic information includes a fingertip fixedpoint, palm centroid, a hand joint, and the like. Features of this typeare usually obtained by using single frame data. Dynamic characteristicinformation includes a displacement vector, a moving speed, and thelike. Such characteristic information is usually obtained by usingmulti-frame data. Because the foregoing is a mature conventionaltechnology, details are not described in this disclosure document.Compared with the conventional technology, the sensor system in thisdisclosure includes a depth sensor. The depth sensor included in thesensor system may be sensor hardware dedicated for depth detection or afunctional software module for depth detection, to obtain depthinformation of a hand, and integrate into user-interface interaction.The sensor system may further store some specific program instructions.A memory 1405, configured to: store the program and various data, andmainly store software units such as an operating system, an application,and a function instruction, or a subset thereof, or an extension setthereof. The memory 1405 may further include a non-volatile randomaccess memory that provides the processor 1401 with hardware, softwareand data resources for managing a computing and processing device andthat supports control software and applications, and is furtherconfigured to store a multimedia file, a running program, and anapplication.

A display element 1403, generally including a display and a supportingoptical component, and configured to display content. A displayinterface is usually presented on a display, to perform man-machineinteraction and file browsing.

An acoustic element 1404, including a microphone, a loudspeaker, aheadset, or the like, and configured to output a sound.

Physical hardware 1406, including a physical function button such as anon/off button, a volume button, or a mechanical control button.

In addition to the foregoing parts 1401 to 1405, the device may furtherinclude another component 1407, configured to enrich functions of thedevice and beautify appearance of the device.

The foregoing hardware 1401 to 1407 may be electrically connectedthrough the bus 1408 to implement coupling and communication.

FIG. 15 is a block diagram of a software structure of a terminalaccording to an embodiment of this disclosure.

In a layered architecture, software is divided into several layers, andeach layer has a clear role and task. The layers communicate with eachother through a software interface. In some embodiments, the Androidsystem is divided into four layers from top to bottom: an applicationlayer, an application framework layer, an Android runtime and a systemlibrary, and a kernel layer.

The application layer may include a series of application packages.

As shown in FIG. 15 , the application packages may include applicationssuch as Phone, Camera, Gallery, Calendar, Call, Map, Navigation, WLAN,Bluetooth, Music, Videos, Messages, and a VR glasses application. The VRglasses application includes a 3D background drawing module, a gestureinformation management module, an application icon loading module, avirtual screen management module, and a virtual screen content obtainingmodule.

The 3D background drawing module is configured to complete drawing of abackground picture displayed in a 3D virtual environment, so that theuser can feel like being in a real scenario.

The gesture information management module is configured to obtaingesture information of the user, so that the user can control a controlon a display interface in the virtual environment based on a gesture.

The application icon loading module is configured to load and display,in the virtual environment of the pair of VR glasses, icons of severalapplications (for example, WeChat, Weibo, and TikTok) on the terminal.

The virtual screen management module is configured to: create a virtualscreen when the user taps an application icon to start an application,and destroy the virtual screen when the user closes the application.

The virtual screen content obtaining module is configured to: when theuser taps a started application, obtain content in the application, andrender the content in the application through distortion, to display thecontent in the virtual environment.

The application framework layer provides an application programminginterface (application programming interface, API) and a programmingframework for an application at the application layer. The applicationframework layer includes some predefined functions.

As shown in FIG. 15 , the application framework layer may include awindow manager, a content provider, a view system, a phone manager, aresource manager, a notification manager, and the like.

The window manager is configured to manage a window program. The windowmanager may obtain a size of the display, determine whether there is astatus bar, perform screen locking, take a screenshot, and the like.

The content provider is configured to: store and obtain data, and enablethe data to be accessed by an application program. The data may includea video, an image, an audio, calls that are made and answered, abrowsing history and bookmarks, an address book, and the like.

The view system includes visual controls such as a control fordisplaying a text and a control for displaying an image. The view systemmay be configured to construct an application program. A displayinterface may include one or more views. For example, a displayinterface including an SMS message notification icon may include a textdisplay view and an image display view.

The phone manager is configured to provide a communication function forthe terminal, for example, management of call statuses (includinganswering, declining, and the like).

The resource manager provides various resources such as a localizedcharacter string, an icon, an image, a layout file, and a video file foran application program.

The notification manager enables an application program to displaynotification information in a status bar, and may be configured toconvey a notification message. The notification manager mayautomatically disappear after a short pause without requiring a userinteraction. For example, the notification manager is configured tonotify download completion, give a message notification, and the like.The notification manager may alternatively be a notification thatappears in a top status bar of the system in a form of a graph or ascroll bar text, for example, a notification of an application that isrun on a background, or may be a notification that appears on the screenin a form of a dialog window. For example, text information is promptedin the status bar, an alert sound is played, the terminal vibrates, andthe indicator light blinks.

In this embodiment of this disclosure, an activity manager service(AMS), a window manager service (WMS), and a download manager service(DMS) in the application framework layer may further include anapplication keep-alive module, an event injection module, and a virtualscreen management module.

The application keep-alive module is configured to: after an applicationthat has a multi-screen display mode function is started, control theterminal to enter the VR multi-screen display mode. In this mode, theterminal may run the plurality of applications at the same time, andsupport the applications to be in the active state at the same time.

The event injection module is configured to: in the multi-screen displaymode, obtain an operation-corresponding event of the user, anddistribute the event to a virtual screen corresponding to anapplication.

The virtual screen management module is configured to provide theterminal with a capability of creating a virtual screen and destroying avirtual screen.

The Android runtime includes a kernel library and a virtual machine. TheAndroid runtime is responsible for scheduling and management of theAndroid system.

The kernel library includes two parts: a function that needs to becalled in Java language and a kernel library of Android.

The application layer and the application framework layer run on thevirtual machine. The virtual machine executes Java files of theapplication layer and the application framework layer as binary files.The virtual machine is configured to implement functions such as objectlifecycle management, stack management, thread management, security andexception management, and garbage collection.

The system library may include a plurality of functional modules, forexample, a surface manager, a media library, a three-dimensionalgraphics processing library (for example, OpenGL ES), and a 2D graphicsengine (for example, SGL).

The surface manager is configured to manage a display subsystem andprovide fusion of 2D and 3D layers for a plurality of applications.

The media library supports playback and recording in a plurality ofcommonly used audio and video formats, and static image files. The medialibrary may support a plurality of audio and video coding formats suchas MPEG-4, H.264, MP3, AAC, AMR, JPG, and PNG.

The three-dimensional graphics processing library is configured toimplement three-dimensional graphics drawing, image rendering,composition, layer processing, and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The kernellayer includes at least a display driver, a camera driver, an audiodriver, and a sensor driver.

It may be clearly understood by a person skilled in the art that, forthe purpose of convenient and brief description, for a detailed workingprocess of the foregoing system, apparatus, and unit, refer to acorresponding process in the foregoing method embodiments, and detailsare not described herein again.

In the several embodiments provided in this disclosure, it should beunderstood that the disclosed system, apparatus, and method may beimplemented in other manners. For example, the described apparatusembodiment is merely an example. For example, division into the units ismerely logical function division and may be other division in actualimplementation. For example, a plurality of units or components may becombined or integrated into another system, or some features may beignored or not performed. In addition, the displayed or discussed mutualcouplings or direct couplings or communication connections may beimplemented by using some interfaces. The indirect couplings orcommunication connections between the apparatuses or units may beimplemented in electronic, mechanical, or other forms.

The units described as separate parts may or may not be physicallyseparate, and parts displayed as units may or may not be physical units,may be located in one position, or may be distributed on a plurality ofnetwork units. Some or all of the units may be selected based on actualrequirements to achieve the objectives of the solutions of embodiments.

In addition, functional units in embodiments of this disclosure may beintegrated into one processing unit, or each of the units may existalone physically, or two or more units are integrated into one unit. Theintegrated unit may be implemented in a form of hardware, or may beimplemented in a form of a software functional unit.

When the integrated unit is implemented in the form of the softwarefunctional unit and sold or used as an independent product, theintegrated unit may be stored in a computer-readable storage medium.Based on such an understanding, the technical solutions of thisdisclosure essentially, or the part contributing to the prior art, orall or some of the technical solutions may be implemented in the form ofa software product. The computer software product is stored in a storagemedium and includes several instructions for instructing a computerdevice (which may be a personal computer, a server, or a network device)to perform all or some of the steps of the methods described inembodiments of this disclosure. The foregoing storage medium includesany medium that can store program code, for example, a USB flash drive,a removable hard disk, a read-only memory (ROM), a random access memory(RAM), a floppy disk, or a compact disc.

The foregoing embodiments are merely intended for describing thetechnical solutions of this disclosure other than limiting thisdisclosure. Although this disclosure is described in detail withreference to the foregoing embodiments, persons of ordinary skill in theart should understand that they may still make modifications to thetechnical solutions described in the foregoing embodiments or makeequivalent replacements to some technical features thereof, withoutdeparting from the spirit and scope of the technical solutions ofembodiments of this disclosure.

1. An interface control method, comprising: obtaining, by a terminal, a first operation on a first interface displayed in a virtual environment, wherein the first interface comprises at least two dividable components; and displaying, by the terminal in the virtual environment based on division location information in response to the first operation, a first subinterface and a second subinterface that are obtained by dividing the first interface, wherein the first subinterface comprises a first dividable component of the at least two dividable components, the second subinterface comprises a second dividable component adjacent to the first dividable component, and the division location information comprises a boundary between the first dividable component and the second dividable component.
 2. The method according to claim 1, wherein the first operation comprises first voice information obtained by the terminal using a voice collection apparatus, or a first gesture obtained by the terminal using a sensor system.
 3. The method according to claim 2, wherein the first gesture comprises: moving a flat palm with fingers together by a preset distance in a first direction in which a palm plane extends, wherein the first direction in which the palm plane extends is parallel to the boundary.
 4. The method according to claim 2, wherein the method further comprises: displaying, by the terminal, a division line on the boundary between the first dividable component and the second dividable component.
 5. The method according to claim 1, wherein displaying, by the terminal in the virtual environment based on division location information in response to the first operation, the first subinterface and the second subinterface that are obtained by dividing the first interface comprises: displaying, by the terminal, the first subinterface in a first location, and displaying the second subinterface in a second location, wherein the first location is the same as a first initial location, a distance between the second location and a second initial location in a direction away from the first initial location is a first preset value, the first initial location is a location of the first subinterface on the first interface when the terminal obtains a first instruction, and the second initial location is a location of the second subinterface on the first interface when the terminal obtains the first instruction; or the second location is the same as a second initial location, and a distance between the first location and a first initial location in a direction away from the second initial location is a second preset value; or a distance between the first location and a first initial location in a direction away from a second initial location is a third preset value, and a distance between the second location and the second initial location in a direction away from the first initial location is a fourth preset value.
 6. The method according to claim 1, wherein the method further comprises: moving, by the terminal, the first subinterface in response to a second operation on the first subinterface.
 7. The method according to claim 6, wherein the second operation comprises: a pinch grip gesture pointing to the first subinterface, wherein the pinch grip gesture comprises a hand state in which a thumb approaches at least one of four fingers, and keeping the pinch grip gesture and moving; or touching and holding the first subinterface and moving; or keeping an open palm gesture and moving, wherein the open palm gesture comprises a hand state with fingers together and a palm up.
 8. The method according to claim 1, wherein the method further comprises: moving, by the terminal, the first subinterface; and based on a distance between the first subinterface and the second subinterface being less than a first threshold, displaying, by the terminal in the virtual environment, a second interface obtained by combining the first subinterface and the second subinterface, wherein a layout of the second interface is the same as or different from a layout of the first interface.
 9. The method according to claim 1, wherein the method further comprises: displaying, by the terminal, a third interface in the virtual environment, wherein the first interface is an interface of a first application, and the third interface is an interface of a second application; moving, by the terminal, the first subinterface and/or the third interface; and based on a distance between the first subinterface and the third interface being less than a second threshold, displaying, by the terminal in the virtual environment, a fourth interface obtained by combining the first subinterface and the third interface.
 10. The method according to claim 1, wherein the method further comprises: displaying, by the terminal, a third subinterface and a fourth subinterface in the virtual environment, wherein the third subinterface and the fourth subinterface are obtained by dividing the third interface, the first interface is the interface of the first application, and the third interface is the interface of the second application; moving, by the terminal, the first subinterface and/or the third subinterface; and based on a distance between the first subinterface and the third subinterface being less than a third threshold, displaying, by the terminal in the virtual environment, a fifth subinterface obtained by combining the first subinterface and the third subinterface.
 11. The method according to claim 10, wherein the first subinterface comprises a first icon for controlling a first function of the first application, the third subinterface comprises a second icon for controlling a second function of the second application, and the first function is the same as the second function; and the fifth subinterface comprises a third icon, and the third icon is used to control the first function of the first application and the second function of the second application.
 12. The method according to claim 10, wherein the first subinterface is an input interface of the first application, the second subinterface is an output interface of the first application, the third subinterface is an input interface of the second application, the fourth subinterface is an output interface of the second application, and the method further comprises: displaying, by the terminal, an indication identifier on an edge of the fifth subinterface, wherein the indication identifier is used to determine an application currently controlled by the fifth subinterface; based on the indication identifier pointing to the second subinterface, controlling, by the terminal, the first application based on an input operation performed on the fifth subinterface; and based on the indication identifier pointing to the fourth subinterface, controlling, by the terminal, the second application based on an input operation performed on the fifth subinterface.
 13. A terminal, comprising one or more processors and a memory, wherein the memory stores computer-readable instructions; and upon the one or more processors reading the computer-readable instructions, the terminal is enabled to implement: obtaining, by the terminal, a first operation on a first interface displayed in a virtual environment, wherein the first interface comprises at least two dividable components; and displaying, by the terminal in the virtual environment based on division location information in response to the first operation, a first subinterface and a second subinterface that are obtained by dividing the first interface, wherein the first subinterface comprises a first dividable component of the at least two dividable components, the second subinterface comprises a second dividable component adjacent to the first dividable component, and the division location information comprises a boundary between the first dividable component and the second dividable component.
 14. The terminal according to claim 13, wherein the first operation comprises first voice information obtained by the terminal using a voice collection apparatus, or a first gesture obtained by the terminal using a sensor system.
 15. The terminal according to claim 14, wherein the first gesture comprises: moving a flat palm with fingers together by a preset distance in a first direction in which a palm plane extends, wherein the first direction in which the palm plane extends is parallel to the boundary.
 16. The terminal according to claim 14, wherein the terminal is further enabled to implement: displaying, by the terminal, a division line on the boundary between the first dividable component and the second dividable component.
 17. The terminal according to claim 13, wherein displaying, by the terminal in the virtual environment based on division location information in response to the first operation, the first subinterface and the second subinterface that are obtained by dividing the first interface specifically comprises: displaying, by the terminal, the first subinterface in a first location, and displaying the second subinterface in a second location, wherein the first location is the same as a first initial location, a distance between the second location and a second initial location in a direction away from the first initial location is a first preset value, the first initial location is a location of the first subinterface on the first interface when the terminal obtains a first instruction, and the second initial location is a location of the second subinterface on the first interface when the terminal obtains the first instruction; or the second location is the same as a second initial location, and a distance between the first location and a first initial location in a direction away from the second initial location is a second preset value; or a distance between the first location and a first initial location in a direction away from a second initial location is a third preset value, and a distance between the second location and the second initial location in a direction away from the first initial location is a fourth preset value.
 18. The terminal according to claim 13, wherein the terminal is further enabled to implement: moving, by the terminal, the first subinterface in response to a second operation on the first subinterface.
 19. The terminal according to claim 18, wherein the second operation comprises: a pinch grip gesture pointing to the first subinterface, wherein the pinch grip gesture comprises a hand state in which a thumb approaches at least one of four fingers, and keeping the pinch grip gesture and moving; or touching and holding the first subinterface and moving; or keeping an open palm gesture and moving, wherein the open palm gesture comprises a hand state with fingers together and a palm up.
 20. The terminal according to claim 13, wherein the terminal is further enabled to implement: moving, by the terminal, the first subinterface; and when a distance between the first subinterface and the second subinterface is less than a first threshold, displaying, by the terminal in the virtual environment, a second interface obtained by combining the first subinterface and the second subinterface, wherein a layout of the second interface is the same as or different from a layout of the first interface. 